Skip to content

Add multi-line REPL input with parse-driven and backslash continuation#88

Merged
mkrueger merged 8 commits into
mainfrom
dev/mkrueger/multi-line-input
May 20, 2026
Merged

Add multi-line REPL input with parse-driven and backslash continuation#88
mkrueger merged 8 commits into
mainfrom
dev/mkrueger/multi-line-input

Conversation

@mkrueger
Copy link
Copy Markdown
Contributor

@mkrueger mkrueger commented May 20, 2026

Adds multi-line REPL input so commands can span more than one physical line.

Fixes AzureCosmosDB/cosmosdb-shell-preview#20: typing \ at end of line + Enter previously produced an error instead of continuing the command.

What changes

Two complementary continuation mechanisms, plus a small parser-signal refinement that made them feasible:

  • Parse-driven continuation. New ShellInterpreter.IsIncompleteInput runs the lexer/parser over the buffered text and treats input as incomplete whenever the parser reports UnexpectedEnd or UnterminatedString. That lets the REPL automatically keep reading lines inside an unclosed { }, a quoted string, or a half-finished statement.
  • Explicit backslash continuation. Bash-style \<newline> continuation. Odd/even backslash runs are handled correctly so \\ at end of line is still a literal pair.
  • ParseError.Kind. New ParseErrorKind enum (Generic, UnexpectedEnd, UnterminatedString). The 17 statement_error_unexpected_end* sites in StatementParser and the missing-close-brace branch now tag themselves UnexpectedEnd; Lexer.ReadDoubleQuotedString/ReadSingleQuotedString/ReadInterpolatedString now record an UnterminatedString error on EOF (new Fluent key lexer_error_unterminated_string). Previously these strings were silently truncated, which is fixed here.

REPL loop behavior

  • ShellInterpreter holds a StringBuilder pendingMultiLineBuffer plus a CosmosShellPrompt reference. While accumulating, the prompt renders ... in grey.
  • Ctrl+C and a cancelled ReadLine both discard the buffer and clear continuation mode.
  • Command history encodes multi-line entries with \\ and \n escapes and decodes on load. Single-line entries are unchanged, so existing history files stay compatible.

Tests

  • Adds MultiLineInputTests covering unclosed brace, unterminated single- and double-quoted strings, balanced input, and empty input.
  • Full Release suite: 970 passed / 0 failed / 74 skipped.

"" continuation:
image

The REPL previously executed every Enter as a complete command, which forced users to fit conditionals, blocks, and quoted strings on a single line. This change introduces two complementary continuation mechanisms so a logical command can span multiple physical lines.

Parse-driven continuation: a new ShellInterpreter.IsIncompleteInput helper runs the lexer and parser over the buffered text and treats it as incomplete whenever an error has Kind UnexpectedEnd or UnterminatedString. To make that signal reliable, ParseError gains a ParseErrorKind enum (Generic, UnexpectedEnd, UnterminatedString); the seventeen statement_error_unexpected_end* call sites in StatementParser now tag themselves UnexpectedEnd, the missing close-brace branch tags itself UnexpectedEnd when no further token is available, and Lexer.ReadDoubleQuotedString, ReadSingleQuotedString, and ReadInterpolatedString record an UnterminatedString ParseError on EOF exit (new Fluent key lexer_error_unterminated_string). The earlier code silently truncated unterminated strings, so this also fixes a quiet correctness gap.

Explicit backslash continuation: bash-style trailing-backslash continuation is recognized in the REPL loop with proper handling of odd/even backslash runs so '\\\\' at end-of-line stays a literal.

Loop state: ShellInterpreter holds a StringBuilder pendingMultiLineBuffer plus a CosmosShellPrompt reference. While accumulating, the prompt renders '...' in grey. Ctrl+C and a cancelled ReadLine both discard the buffer and clear continuation mode. Command history saves multi-line entries with backslash-encoded newlines (\\\\ and \\n) and decodes on load; single-line entries are unchanged so existing history files stay compatible.

Adds MultiLineInputTests covering unclosed brace, unterminated single/double-quoted strings, balanced input, and empty input. Full suite: 927 passed / 0 failed / 74 skipped.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the interactive Cosmos DB Shell REPL to support multi-line command entry, using both explicit \<newline> continuation and parse-driven continuation based on lexer/parser signals. It also refines parser error categorization to better distinguish “incomplete input” from definitive syntax errors, and updates command history persistence to handle multi-line entries.

Changes:

  • Added REPL multi-line buffering with a continuation prompt (...) plus explicit backslash continuation handling.
  • Introduced ParseErrorKind and tagged key “unexpected end” / “unterminated string” paths so the REPL can detect incomplete input.
  • Implemented history line encoding/decoding for multi-line commands and added unit tests for the new helpers/behaviors.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
CosmosDBShell/lang/en.ftl Adds a new localized lexer error string for unterminated string literals.
CosmosDBShell/Azure.Data.Cosmos.Shell.Parser/StatementParser.cs Tags end-of-input parsing errors with ParseErrorKind.UnexpectedEnd to enable parse-driven continuation.
CosmosDBShell/Azure.Data.Cosmos.Shell.Parser/ParseError.cs Introduces ParseErrorKind and adds Kind to ParseError for REPL continuation decisions.
CosmosDBShell/Azure.Data.Cosmos.Shell.Parser/Lexer.cs Detects unterminated string literals at EOF and records them as ParseErrorKind.UnterminatedString.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Implements multi-line REPL loop behavior, prompt continuation state, and history encoding/decoding.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/CosmosShellPrompt.cs Renders a continuation prompt (...) while multi-line input is being accumulated.
CosmosDBShell.Tests/Shell/MultiLineInputTests.cs Adds tests covering incomplete detection, backslash continuation, history round-tripping, and append behavior.

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Parser/ParseError.cs
Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs
Copilot AI review requested due to automatic review settings May 20, 2026 11:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
Copilot AI review requested due to automatic review settings May 20, 2026 11:36
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs
Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
Copilot AI review requested due to automatic review settings May 20, 2026 11:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

@mkrueger mkrueger enabled auto-merge May 20, 2026 11:59
@mkrueger mkrueger merged commit 83049be into main May 20, 2026
9 checks passed
@mkrueger mkrueger deleted the dev/mkrueger/multi-line-input branch May 20, 2026 12:10
mkrueger added a commit that referenced this pull request May 21, 2026
Converts the `Unreleased — since v1.0.273` section to `1.1.4 —
2026-05-21` (first release on the 1.1 line) and adds entries for the PRs
that landed after #86 (the last CHANGELOG update):

- **Highlights / New features:** multi-line REPL input with `\`
continuation and parser-driven incomplete-input detection
([#88](#88)); parser & query
diagnostics with line, column, source caret, and "Did you mean…"
suggestions ([#87](#87)).
- **Documentation:** notes that [docs/navigation.md](docs/navigation.md)
and [README](README.md) now document multi-line input.
- **Build & pipeline:** versioning moved to
[Nerdbank.GitVersioning](https://github.com/dotnet/Nerdbank.GitVersioning);
local builds now produce the same versions as CI
([#90](#90),
[#91](#91)).

PR #89 was a stepping stone superseded by the NBGV migration, so it
isn't called out separately.

Docs-only change — no code touched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can we do multiline commands?

3 participants